A touch of chaos can help artificial intelligence learn faster

• The study, published in Physical Review Research, shows that neural networks can train faster when their learning dynamics operate at the onset of chaotic behavior, balancing exploration and exploitation during optimization.
• The study was carried out by researchers at the Institute for Cross-Disciplinary Physics and Complex Systems (IFISC, CSIC-UIB), applying concepts from chaos theory and complex systems to understand how artificial neural networks learn.

Training a neural network to perform a task, such as making predictions or classifications—consists of smoothly and gradually adjusting its internal parameters. Once its performance reaches a satisfactory level, we say that the network has learned the task. . New research from IFISC shows that introducing a controlled amount of chaos into this process can actually accelerate learning.

In the study published in Physical Review Research, researchers found that artificial neural networks can train more efficiently when their learning dynamics operate near the onset of chaotic behavior. At this point, the system balances two complementary strategies: refining known solutions and exploring new possibilities in the vast space of possible network configurations.

The “edge of chaos” as a driver of efficiency

Artificial neural networks typically learn using optimization algorithms such as gradient descent, which gradually adjust the model's parameters to reduce errors. The learning rate acts like the step size of these adjustments: small values ensure cautious, stable progress toward a solution; larger ones take bolder leaps that risk overshooting. This process is generally stable and exploitative, steadily refining the current solution, like a hiker following a well-marked trail. But when the learning rate grows large, IFISC researchers found that training dynamics become sensitive to tiny differences in starting points, a hallmark of chaos: two nearly identical neural networks can diverge dramatically over the action of learning, like butterflies whose wing flaps spawn hurricanes thousands of miles away.

"Instead of harming learning, this chaotic instability can actually accelerate it", explains Lucas Lacasa, researcher at IFISC and co-author of the study. "Near the boundary where chaotic dynamics begin, the system finds a sweet spot that allows it to learn significantly faster”.

The researchers tracked the "paths" that network parameters follow during training and measured how sensitive they are to starting points. For small learning rates, everything flows smoothly and orderly; with huge values, total chaos causes learning to collapse. But right in that intermediate zone, where exploration and exploitation balance out, networks learn accurate representations, and training becomes surprisingly faster.

Toward faster and more efficient AI

The phenomenon was observed across different neural network architectures, activation functions, and datasets, suggesting that it may represent a robust feature of learning dynamics in the systems they studied.

"The accelerated training we observe near the edge of stability turns out to be remarkably robust" says Miguel C. Soriano, researcher at IFISC and co-author of the study. "It consistently appears across the different architectures, activation functions, and datasets we tested".

Beyond its potential practical implications for accelerating training, the findings also connect modern machine learning with the "edge of chaos" hypothesis from complex systems science, which proposes that systems capable of computation often perform optimally at the boundary between order and disorder.

"Our results suggest that, for the neural networks we studied, learning is most efficient precisely near this edge of chaos," says first author Pedro Jiménez-González. "Understanding and exploiting this regime could help design faster and more efficient AI systems in the future".

Image: Neural network training can be seen as a trajectory through a high-dimensional graph space. Along these trajectories, the connections between neurons evolve over time. These trajectories can become chaotic for certain regimes.

Jiménez-González, P., Soriano, M. C., and Lacasa, L. (2026). Leveraging chaotic transients in the training of artificial neural networks. Physical Review Research. https://doi.org/10.1103/t5p9-kv5w

• CSIC
• Xornal Galicia
• UIB